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DETAILED ACTION 

1 . This Office Action is in response to communications filed on 3/30/2009. Claims 1 
and 3-10 are pending in this application. 

Response to Arguments 

2. The Applicant's arguments with regards to the claims have been fully considered 
but are moot in view of the new ground(s) of rejection. 

Drawings 

3. The drawings are objected to because Figures 19 and 20 should be labeled 
'Prior Art' as they illustrate a conventional imaging apparatus and pixel shifting 
technique Corrected drawing sheets in compliance with 37 CFR 1 .121(d) are required 
in reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure 
number of an amended drawing should not be labeled as "amended." If a drawing figure 
is to be canceled, the appropriate figure must be removed from the replacement sheet, 
and where necessary, the remaining figures must be renumbered and appropriate 
changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering 
of the remaining figures. Each drawing sheet submitted after the filing date of an 
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application must be labeled in the top margin as either "Replacement Sheet" or "New 
Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
the applicant will be notified and informed of any required corrective action in the next 
Office action. The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1 and 3-5 are rejected under 35 U.S.C 103 as being unpatentable 
over Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] in view of Applicant's 
Admitted Prior Art (US PGPub 2007/0159535) [hereafter AAP A]. 

6. As to claim 1 , Nonaka teaches a multi-eye imaging apparatus (Fig. 1 ) that 
comprises a plurality of imaging systems comprising optical systems (1 land 16) and 
imaging elements (12 and 17) that have different optical axes, wherein the imaging 
systems include a first imaging system (11,12, and 21) having pixel shifting means of 
an actuator (21 ) for changing a relative positional relationship between an image of a 
subject (1 1 0) formed on the imaging element (1 2) and the imaging element, and a 
second imaging system (16 and 17) in which a relative positional relationship between 
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an image of a subject (110) formed on the imaging element (17) and tlie imaging 
element is fixed during time-series image capture (Col. 3, 29-53, Col. 4, 47-59). 

Additionally, Nonaka discloses an image memory (19a) for accumulating a 
plurality of frames of image information captured in time series (Fig. 2, 10, II) contained 
within a camera shake detection section (19) that compares the plurality of frames of 
image information accumulated in the image memory section with a comparison section 
(19b), moving direction calculation section (19c), and a moving amount calculation 
section (19d), and an image processing/forming section (22) that synthesizes the image 
signals contained within the image memory sections upon one another (Fig. 5, 24b), 
corresponding to combining the plurality of frames of images accumulated in the image 
memory (Col. 4, 9-46, Col. 6, 26-34). 

It is however noted that Nonaka fails to disclose combining images after pixels 
are shifted relative to each other so the resolution of the combined image is higher than 
that of the plurality of frames of images. 

On the other hand, AAPA discloses a technique called "pixel shift" (Fig. 20A- 
20C) for improving the resolution of an imaging apparatus. The technique involves 
shifting (Fig. 20B) the active region (2101) of an imaging element (P) to a different 
position relative to an initial position (Fig. 20A) in order to pick-up additional image data 
of a scene not originally captured by the imaging element. The images captured (pre- 
shift and post-shift) are combined and result in an image whose resolution is equivalent 
to an image captured using an imaging element that is doubled in size (Page 1, 0009- 
0012). 
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It would have been obvious to one having ordinary skill in the art at the time of 
invention to incorporate combining images after pixels are shifted relative to each other 
so the resolution of the combined image is increased as disclosed by AAPA to increase 
the resolution of the multi-eye imaging apparatus equipped with a pixel shifting means 
and image combining means of Nonaka because the prior art are directed towards 
imaging devices containing movable imaging elements used for capturing images of a 
scene and because the technique of combining images after pixels have been shifted in 
order to increase a resolution of an imaging element was recognized as part of the 
ordinary capabilities of one skilled in the art that would yield predictable a result of 
increasing the resolution of images captured by the multi-eye imaging apparatus to a 
resolution higher than that of the plurality of image frames that are synthesized by the 
image processing/forming section (22) of Nonaka. 

7. As to claim 3, Nonaka teaches the change amount of the positional relationship 
by the actuator is determined based on the shake amount obtained by the shake 
detection section (19) (Col. 4, 29-59). 

8. As to claim 4, AAPA discloses the pixels of the imaging element are shifted by 
1/2 of a pixel in both the horizontal and vertical directions (Fig. 20B) (Page 1 , 0010). 

9. As to claim 5, the Nonaka reference discloses all claimed subject matter with 
regards to similar comments of claims 1 and 2. Additionally, Nonaka teaches a 
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calculation control section (25) that calculates a magnitude of a parallax from images 
captured by the various imaging systems (Col. 7, 63-67, Col. 8, 1-6). 

1 0. Claims 6 and 10 are rejected under 35 U.S.C 103 as being unpatentable 
over Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] and Applicant's 
Admitted Prior Art (US PGPub 2007/0159535) [hereafter A AP A], as applied to 
claim 5, in view of Tamamura (US Patent 7,463,284) [hereafter Tamamura] filed on 
11/15/2005. 

11. As to claim 6, it is noted that Nonaka and AAPA fail to teach an optimal image 
selecting means for selecting image information which is used in the combination of the 
image combining means based on the shake amount obtained by shake amount 
obtaining means and parallax amount obtained by parallax amount obtaining means. 

On the other hand, Tamamura teaches a camera (Fig. 1) that combines images 
using an image composition unit (117) in order to form images free from image blur due 
to camera shake. Images captured by an image sensing unit (15) are processed by an 
image processor (112) and feature points of the image are extracted by a shift detector 
(1 1 3) and the coordinates of the extracted feature points are determined by a 
coordinate converter (114), the coordinates determining shift amounts between images. 
The images are stored within an image storage unit (115) and combinations of stored 
images are selected by a selector (116) and subsequently combined by the image 
composition unit. The size of the image comprising the combination of two previously 
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stored images is calculated by an image size unit (118) and the calculated sizes for the 
synthesized images are compared by a comparator (120) that selects a synthesized 
image having the greatest size, corresponding to the least amount of shake between 
the two images, and outputs the image with the least amount of shake to recording and 
display units (122 and 121) (Col. 3, 13-18, Col. 4, 3-41). 

It would have been obvious to one having ordinary skill in the art at the time of 
invention to include a comparator to select image information to be used in a selected 
combination of image signals based on a detected amount of variation between the 
stored images as taught by Tamamura with the multi-eye Imaging apparatus of Nonaka, 
modified with the teachings of AAPA, because the prior art are directed towards imaging 
devices that eliminate image shaking effects and because it would allow the device of 
Nonaka to obtain a more accurate position displacement calculation for an image 
sensor to be moved by an actuator by further analyzing camera shake after an initial 
compensation value for the movement of the actuator has been calculated and the 
image sensor has consequentially been moved, and incorporating the parallax amount 
into the decision as to which combination of images is to be used would allow the 
device of Nonaka to increase the accuracy of the position displacement calculation for 
an image sensor by taking into account the distance between images formed on the 
various image sensors. 



12. As to claim 10, the Nonaka, AAPA, and Tamamura references disclose all 
claimed subject matter with regard to the comments of claims 4-6. 
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1 3. Claims 7-8 are rejected under 35 U.S.C 103 as being unpatentable over 
Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] and Applicant's Admitted 
Prior Art (US PGPub 2007/0159535) [hereafter AAP A], as applied to claim 1, in 
view of Nakazono (Japanese Patent Publication 2003-134385) [hereafter 
Nakazono] published on 5/9/2003. 

14. As to claim 7, it is noted tliat Nonal<a and AAPA fail to disclose means for 
discriminating different subjects wherein the shake amount obtaining means obtains a 
shal<e amount for each of the different subjects and the image combining means 
combines images for each of the different subjects. 

On the other hand, Nakazono teaches a camera (Fig. 1) that captures an image 
using a CCD imager (1) and using an image composition device (4) that comprises a 
motion vector detecting element (Fig. 2, 1 1 , Fig. 3) that uses two pictures to 
discriminate subject images (Fig. 5) and determines a shake amount for the subject 
using a motion vector calculation part (Fig. 3, 23) and the two images used to determine 
a shake amount of the subject of the image are combined (Fig. 15) into an output image 
using a synthesizing means (Fig. 14, 92) (Page 1 1 , 0056-0058, Page 12, 0059-0060, 
Page 16, 0073-0075). 

It would have been obvious to one having ordinary skill in the art at the time of 
invention to discriminate subjects of an image and obtain shaking amounts of the 
identified subjects as taught by Nakazono with the multi-eye imaging apparatus of 
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Nonaka, modified with the teachings of AAPA, because the prior art are directed 
towards imaging devices that eliminate image shaking effects and because obtaining 
shake amounts, or movement amounts, of an imaged subject between two images 
would allow the device of Nonaka to determine an image shake amount for an image's 
subject as opposed to the entire image in order to compensate for an image where the 
subject itself is moving as opposed to the physical camera, resulting in an image free 
from shaking or blurring effects. 

1 5. As to claim 8, the Nakazono reference discloses all claimed subject matter with 
regards to similar comments of claim 7. Additionally, Nakazono teaches dividing image 
information into a plurality of blocks (Fig. 8) and obtaining a shake amount for a plurality 
of blocks using block setting means (Fig. 3, 22) (Page 13, 0064-0065). 

1 6. Claim 9 is rejected under 35 U.S.C 103 as being unpatentable over Nonaka 
et al. (US Patent 7,162,151) [hereafter Nonaka] and Applicant's Admitted Prior Art 
(US PGPub 2007/0159535) [hereafter AAPA], as applied to claim 1, in view of Yu et 
al. (US Patent 6,611,289) [hereafter Yu] published on 8/26/2003. 

1 7. As to claim 9, Nonaka teaches the optical photographing lens (1 1 ) forms a main 
image of a subject (110) onto an image sensor (12), which captures all incident light and 
corresponding colors, and that light receiving lens (16) receives reflected luminous flux 
from the subject and an image sensor (17), which also captures all incident light and 
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corresponding colors, is used to form images based on the reflected luminous flux and 
subsequently these images are used to determine moving direction of the image sensor 
(12) (Col. 3, 42-53, 66-67, Col. 4, 1-8, 20-28). 

It is however noted that Nonaka and AAPA fail to disclose an imaging system for 
handling a red color, an imaging system for handling a green color, and an imaging 
system for handling a blue color, wherein, for at least one corresponding to one color of 
the imaging systems corresponding to the respective colors, the number of the imaging 
systems corresponding to the one color is two or more. 

On the other hand, Yu teaches a camera (Fig. 3) with a plurality of imaging 
systems having different optical axes that are composed of an imaging system 
comprising of a lens (310) and an image sensor (302) for handling a red color, a lens 
(312) and an image sensor (304) for handling a green color, a lens (314) and an image 
sensor (306) for handling a blue color, and a lens (316) and an image sensor (308) for 
handling all colors of the visible light spectrum including the colors of the other imaging 
systems (Col. 4, 62-67, Col. 5, 1-39, 65-66, Col. 8, 59-61). 

It would have been obvious to one having ordinary skill in the art at the time of 
invention to include separate and distinct imaging systems that each image a different 
RGB color and form a high-quality image and additionally include an imaging system 
that also images each of the RGB colors within a camera as taught by Yu with the multi- 
eye imaging apparatus of Nonaka, modified with the teachings of AAPA, because the 
prior art are directed towards imaging devices that include multiple imaging systems for 
imaging a single object and because such a camera configuration would allow the 
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device of Nonaka to produce true color images witli enlianced resolutions that are free 
from image/camera shake effects by using the RGB color imaging systems of Yu as the 
multi-eye imaging systems (main system 11 and 12, sub-sampling system 16 and 17) 
within the configuration of Nonaka to detect camera shake and subsequently calculate 
movement amounts for the monochrome and multicolor image sensors incorporated 
within the respective imaging systems. 

Conclusion 

18. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael Osinski whose telephone number is (571) 270- 
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3949. The examiner can normally be reached on Monday to Thursday 9 a.m. to 6 p.m. 
If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jason Chan can be reached on (571) 272-3022. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retheval (PAIR) system. Status Information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more Information about the PAIR system, see http://palr-dlrect.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
/Lin Ye/ 

Supervisory Patent Examiner, Art Unit 2622 
MO 



